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Abstract 



Consider a case-control study in which the aim is to assess a factor's effect on dis- 
ease occurrence. We suppose that this factor is dichotomous. Also suppose that 
the data consists of two strata, each stratum summarized by a two-by-two table. 
A commonly-proposed two-stage analysis of this type of data is the following. We 
carry out a preliminary test of homogeneity of the stratum-specific odds ratios. If 
the null hypothesis of homogeneity is accepted then we find a confidence interval 
for the assumed common value (across strata) of the odds ratio. We examine the 
statistical properties of this two-stage analysis, based on the Woolf method, on con- 
fidence intervals constructed for the stratum-specific odds ratios, for large numbers 
of cases and controls for each stratum. We provide both a Monte Carlo simulation 
method and an elegant large-sample method for this examination. These methods 
are applied to obtain numerical results in the context of the large numbers of cases 
and controls for each stratum that arose in a real-life dataset. In this context, we 
find that the preliminary test of homogeneity of the stratum-specific odds ratios has 
a very harmful effect on the coverage probabilities of these confidence intervals. 

Keywords: case-control study; coverage probability; odds ratio; simultaneous con- 
fidence intervals; test of homogeneity. 
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1. Introduction 

Consider a case-control study in which the aim is to assess a factor's effect on disease 
occurrence. We suppose that this factor is dichotomous. Also suppose that the data 
consists of two strata, each stratum summarized by a 2 x 2 table. The parameters 
of interest are the stratum-specific odds ratios. A commonly-proposed two-stage 
analysis of this type of data is the following, see e.g. section 4.4 of Breslow and Day 
(1980), Section 16.2 of Pagano and Gauvreau (2000) and Section 13.6 of Rosner 
(2011). We carry out a preliminary test of homogeneity of the stratum-specific odds 
ratios. If the null hypothesis of homogeneity is accepted then we find a confidence 
interval for the assumed common value (across strata) of the odds ratio. 

Prom a practical point of view, we must state what action we take when the null 
hypothesis of homogeneity of stratum-specific odds ratios is rejected. It would not 
make sense for a consulting statistician to tell a client that this null hypothesis has 
been rejected and so the statistician will do nothing. There is some awareness of the 
need to clearly state what action we take when the null hypothesis of homogeneity 
is rejected, see e.g. p. 279 of Rothman et al (2008) and p. 620 of Rosner (2011). 
The latter states that "If the true ORs are significantly different, then it makes no 
sense to obtain a pooled- OR estimate ... Instead, separate OBs should be reported" . 
We suppose that when the null hypothesis of homogeneity is rejected, we compute 
confidence intervals for each of the stratum-specific odds ratios. 

Our aim is to examine the statistical properties of this two-stage analysis, in the 
context of simultaneous inference for the stratum-specific odds ratios. In Section 2, 
we provide a precise general formulation of this two-stage analysis. We examine the 
statistical properties of this two-stage analysis using the Woolf method (described 
e.g. on p. 139 of Breslow and Day, 1980) to carry out the preliminary test of ho- 
mogeneity of the stratum-specific odds ratios and to construct confidence intervals 
for the stratum-specific odds ratios, for two strata and large numbers of cases and 
controls for each stratum. We provide both a Monte Carlo simulation method and 
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an elegant large-sample method for this examination. These methods are applied to 
obtain numerical results in the context of case and control sample sizes that come 
from a study whose aim is to assess the effect of the consumption of caffeinated coffee 
on nonfatal myocardial infarctions for adult males under the age of 55 (Pagano and 
Gauvreau, 2000 and Rosenberg et al, 1988). Our general conclusion is that the pre- 
liminary test of homogeneity of the stratum-specific odds ratios has a very harmful 
effect on the coverage probabilities of the confidence intervals for these odds ratios, 
for two strata when the numbers of cases and controls in each stratum is large. 

2. Precise general formulation of the two-stage analysis 

For easier cross-referencing with the notation used in Section 3, we phrase our dis- 
cussion in terms of log odds ratios. Let 9i denote the log odds ratio for the i th 
stratum {i — 1,2). To provide a precise general formulation of the two-stage analy- 
sis, our first step is to describe what we would do if it was known with certainty (a) 
that 6*1 7^ 6*2 and (b) that 9i = 62. We consider simultaneous inference for 6*1 and 
02- Consequently, this description is in terms of simultaneous confidence intervals 
for 61 and 62. 

• Suppose that 61 7^ 62 

Use the confidence interval Ii for 9i based solely on the two-by-two table for stra- 
tum 1, with approximate coverage Vl ~ ct. Also, use the confidence interval I2 for 
6*2 based solely on the two-by-two table for stratum 2, with approximate coverage 
^/T^^a. The confidence intervals Ii and I2 have simultaneous coverage approxi- 
mately 1 - a, since P(^i G A, ^2 e ^2) = ^(^1 e A) P{02 e 12) ^ 1 - a. 

• Suppose that 0i — 62 

Let J be the confidence interval for — 6-1 — 92 based on the two-by-two tables for 
both strata, with approximate coverage 1 — a. Let Ji — J and J2 — J he confidence 
intervals for 9i and 6*2, respectively. These confidence intervals have simultaneous 
coverage approximately 1 — a, since P(^i e Ji, ^2 € •^2) = € ~ 1 — 
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The two-stage analysis is precisely formulated as follows. If the null hypothesis of 
homogeneity is rejected then we use the confidence intervals A and /2 for 9i and 6*2, 
respectively. If, on the other hand, this null hypothesis is accepted then we use the 
confidence intervals Ji and J2 for 9i and 62, respectively. The nominal simultaneous 
coverage of the resulting confidence intervals for 6*1 and 6*2 is 1 — a. We will assess 
this two-stage analysis by comparing the actual simultaneous coverage probability of 
these confidence intervals with 1 — a. Of course, there are several different possible 
choices of preliminary test of homogeneity and confidence intervals that can be 
used in the two-stage analysis. As explained in the next section, we use tests and 
confidence intervals based on the Woolf method. 

3. The two-stage analysis that will be evaluated 

We use the following notation for the 2x2 contingency table that summarizes the 
data for the i th stratum (i — 1, 2). Let denote the number of subjects with the 
disease (cases), with j/j of these subjects exposed to the factor. Also, let n'^ denote the 
number of subjects without the disease (controls), with yl of these subjects exposed 
to the factor. We use upper case to denote random variables and lower case to 
denote observed values. Thus, for example, Yi is the random variable corresponding 
to the observed value yj. We use the following model for the data in this table. 
The random variables and Y/ are independent, with Yi ~ Binomial(ni,pj) and 
Y- ~ Binomial(n^,p9. Let e be a specified small positive number (0 < e < |). 
Suppose that Pi e [e, 1 — e] and p[ & [e, 1 — e], for i = 1, 2. The parameter of interest 
for this table is the odds ratio 

We find a confidence interval for as follows. We first find a confidence interval 
for the log odds ratio 6*, = Inlipj) and then transform this in the obvious way into a 
confidence interval for the odds ratio ipi. 
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We consider the two-stage analysis, described in Section 2, implemented using 
Woolf's method. The maximum likelihood estimates of pi and are pi = yi/rii and 
P'i — y'il'^'v respectively. The resulting estimator of Oi is 

' U/(i-Pi);- 

This estimator has a number of disadvantages, including the fact that it is undefined 
for Hi either or rij and for y[ either or n^. We do not use this estimator. Instead, 
we follow the common recommendation (see e.g. page 139 of Breslow and Day, 
1980) of estimating pi and p- by pi = {ui + \)/{ni + 1) and p'^ = {y[ + \)/{n^ + 1), 
respectively. The resulting estimator of 9i is 

This estimator has the following three advantages. Firstly, it is defined for all 
possible values of yi and y[. Secondly, according to page 32 of Cox and Snell (1989), 
©j is an asymptotically less biased estimator of 9i than ©j. Thirdly, the use of 
this type of adjustment of the maximum likelihood estimates of Pi and p[ can be 
remarkably effective in improving the coverage probability properties of Wald-type 
confidence intervals based on these estimates, see e.g. Agresti and Caffo (2000). 
Woolf's method is based on the approximation that 

Qi^N{ei,at), 

where 



(J, 



,2 1 / I 1 \ 1/1 



ni\Pi I- Pi J K \Pi ^-P 
and the approximation that cr| is equal to 

1^1 1\ 1/1 1 



^ • = — I ^ + Z Z- 1 + ^ I T7 + 



Hi \pi ■ I -pi J ' n\ \p[ I- Pi 
We test the null hypothesis of homogeneity Hq : 9i — 02 against the alternative 
hypothesis Hi that the 9i 62. We carry out this test using the test statistic 

01-©2 



T 
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We make the approximation that T ~ A'^(0, 1) under Hq. Let /3 denote the nominal 
level of significance of this test. 

If Hq is rejected then the confidence intervals for 9i and 02, with nominal simul- 
taneous coverage 1 — a, are Ii and 12 respectively, where 

with Ca defined hy P[ — Ca < Z < Ca) = ^J\ — a. for Z ~ iV(0, 1). If, on the other 
hand, Hq is accepted then we carry out inference based on the assumption that 
^1 = ^2 = Q- Define 

e 



which is the estimator of Q assuming that 9i — 02 — 0. Ii 9 — 9i — 02 then the 
following confidence interval for 9 has nominal coverage 1 — a: 



J 



I I \ -1/2 ^ /I T X -1/2- 

erf (7^7 \a 



where Cq, is defined by P ( — c^, < Z < ) = 1 - a for Z ~ iV(0, 1). Let Ji = J and 
J2 = J. If Hq is accepted then the confidence intervals for 9i and 92, with nominal 
simultaneous coverage 1 — a, are Ji and J2, respectively. 

4. Application to case and control sample sizes that arise in 
a real-life data set 

Consider the case-control study data with two strata, described on p. 376 of Pagano 
and Gauvrcau (2000). This data originates from Rosenberg et al (1988). Pagano 
and Gauvreau (2000) carry out a preliminary test of homogeneity of the odds ratios 
for these 2 strata, which is almost identical to that described in the previous section. 
They conclude that they cannot reject the null hypothesis of the odds ratios being the 
same for these 2 strata. They then use the Mantel-Haenszel method to estimate the 
odds ratio, which is assumed to be the same for both of these strata. However, the 
Mantel-Haenszel method is inefficient, in the context of a fixed number of strata and 



large numbers of cases and controls for each stratum, unless special circumstances 
hold (Tarone et al, 1983). This is one of the reasons why we estimate the common 
odds ratio from the two strata in the previous section using Woolf 's method. The 
other reason for doing this is that this permits us to find the elegant large-sample 
approximation described in Section 5. 

For the data described on p. 376 of Pagano and Gauvreau (2000), k — 2, rii — 
1092, n[ = 467, n2 = 449 and ^2 = 488. The parameters of interest are the stratum- 
specific log odds ratios 

Our aim is to find confidence intervals for 6*1 and 6*2 with simultaneous coverage 
1 — a. We suppose that 1 — a — 0.95. We also suppose that (pi,p'i,P2,P2) belongs to 
the set A — [0.02, 0.98]^. Under this restriction, it is expected that the distributions 
of 01 and 02 will be close to normal. To see this, consider the rule-of-thumb 
that the cdf of y ~ Binomial(n, p) is approximated well by the N{np,np{l — p)) 
cdf if np{l — p) > 5 (see e.g. p. 133 of Rosner, 2011). Note that nipi(l — pi) > 
21.403, n[p[{l -p[) > 9.153, n2P2(l - P2) > 8.800 and n^p^(l - p^) > 9.565 for 
all (pi,pi,P2,P2) in A = [0.02,0.98]! Thus, the distributions of Yi, Y(, Y2 and Y^ 
will be close to normal for all (pi,p'i,P2,P2) in A = [0.02,0.98]^. Consequently, we 
expect the distributions of 0i and ©2 to be close to normal. 

Firstly, consider the Woolf method confidence intervals Ii and /2, when we do 
not carry out a preliminary test of homogeneity of the stratum-specific odds ratios. 
Because ni, n[, n2 and n'2 are large, we expect that the simultaneous coverage 
probability P{9i e /i, 62 G I2) will not fall far below 1—a — 0.95 for all (pi,p'i,P2,P2) 
in A. Using the simulation method described in Appendix A, we obtained the rough 
estimate 0.951 of the minimum simultaneous coverage probability. This coverage 
probability is attained at (pi,pi,P2,P2) = (0.596,0.788,0.308,0.788). All of the 
computations presented in this paper were performed with programs written in 
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MATLAB, using the statistics toolbox. 

Now consider the two-stage analysis. Suppose that the nominal level of signif- 
icance of the preliminary hypothesis test is 0.05. Using the simulation method de- 
scribed in Appendix A, we obtained the rough estimate 0.131 of the minimum simul- 
taneous coverage probability. This coverage probability is attained at ipi, p'l, P2, p'2) = 
(0.692, 0.596, 0.02, 0.02). Actually, this estimate is an accurate Monte Carlo simula- 
tion estimate of an upper bound to the minimum simultaneous coverage probability. 
This shows that the confidence intervals resulting from the two-stage analysis are 
completely inadequate. 

5. The large-sample approximation 

Note that ©i and ©2 are independent random variables. It may be proved that (©i — 
9\)/a\ and (©2— ^2)/c2 both converge in distribution to A^(0, 1) (as min(ni, n^, 712, n'2) — >■ 
00). It may also be proved that a"i and a 2 converge in probability to cti and (T2, re- 
spectively (as min(ni, n']^, n2, ^2) — )■ 00). So, the large-sample approximation that 
we will use to analyze the procedure described in the previous section is as follows. 
Firstly, 0, has an N{9i,af) distribution, when both rii and n[ are large (i = 1,2). 
Secondly, in the expressions for 0, T, J and /j (given in the previous section), we 
may replace (Tj by (7j (i = 1, 2). Thirdly, we assume that af and cr| are known. In 
Appendix C, we apply this approximation to obtain a large-sample approximation 
to the simultaneous coverage probability of the confidence intervals for 61 and 62, 
with nominal simultaneous coverage 1 — a, resulting from the two-stage analysis. 

6. Numerical results obtained using the large-sample 
approximation 

The case-control study data described in Section 3 consists of two strata with sample 
sizes rii — 1092, n[ — 467, 712 = 449 and ^2 = 488. In the present section, we 
consider the same number of strata and the same sample sizes. Our aim is to find 
confidence intervals for 6*1 and 6*2 with simultaneous coverage 1 — a. As in Section 
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3, we suppose that {pi,Pi,P2,P2) belongs to the set A = [0.02, 0.98]^. 

Firstly, consider the Woolf method confidence intervals /i and 12, when we do 
not carry out a preliminary test of homogeneity of the stratum-specific odds ra- 
tios. Obviously, the large-sample approximation described in Section 4 tells us that 
P{Oi e /i,^2 e /s) = 1 - a for all (pi,p'i,P2,P2) in A. 

Now consider the two-stage analysis. As in Section 3, suppose that 1 — a — 0.95. 
Also suppose that the nominal level of significance of the preliminary hypothesis 
test is 0.05. Define the step length h = 0.096. The large-sample approximation to 
the coverage probability, described in detail in Appendix C, was computed for each 
iPuP'vP2,P2) belonging to the set (0.02, 0.02+/i, 0.02+2/i, . . . , 0.98)^. The minimum 
value of this large-sample approximation was found to be 0.134847. This value was 
achieved at (pi,p'i,P2,P2) taking any one of the following values: (0.692, 0.596, 0.02, 
0.02), (0.308, 0.404, 0.02, 0.02), (0.692, 0.596, 0.98, 0.98) and (0.308, 0.404, 0.98, 
0.98). The large-sample approximation is a smooth function of {pi,p'i,P2,P2) ^-nd 
so this minimum value can be expected to be an accurate approximation to large 
sample approximation minimized over {pi,p'i,P2,P2) in ^• 

For the two-stage analysis, the simulation estimate of the minimum simultaneous 
coverage probability of the confidence intervals for the stratum-specific odds ratios 
was found to be 0.131. This is quite close to the minimum value of the large-sample 
approximation to the simultaneous coverage probability of these confidence intervals, 
which was found to be 0.134847. In this context, we find that the prehminary test 
of homogeneity of the stratum-specific odds ratios has a very harmful effect on the 
coverage probabilities of these confidence intervals. 

7. The simultaneous coverage probability of the confidence 
intervals resulting from the two-stage analysis is small away 
from the boundaries of the parameter space 

In this section we deal exclusively with the confidence intervals resulting from the 
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two-stage analysis. As noted in the previous sections, the minimum simultaneous 
coverage probability is achieved on the boundary of the parameter space A for both 
the simulation and large-sample estimates of this minimum coverage probability. If 
this simultaneous coverage probability is small only at or near the boundaries of the 
parameter space then it might be argued that statistical practitioners need not be 
concerned about the smallness of the minimum simultaneous coverage probability. 
Therefore, it is natural to ask the question: Is this simultaneous coverage probability 
small only at or near the boundaries of the parameter space? 

In this section, we show that this simultaneous coverage probability is also small 
far from the boundaries of the parameter space A. We do this as follows. Suppose 
that ipi,p'i,P2,P2) belongs to the set A = [e, 1 — e]"^, where e is a small specified 
positive number (0 < e < |). Let 



Also let A' = r A. For each pi in [e A, 1 - e - A] and p[ in [e + A',1- e - A'], 
we find the minimum over P2 — Pi + S and p'2 = p\ — rS, where 5 is in [—A, A], 
of the large-sample simultaneous coverage probability of the confidence intervals for 
9i and 62 resulting from the two-stage analysis. We then examine this partially- 
minimized coverage probability using a contour plot of it, as a function of {pi,p[) 
in [e + A,l - e - A] x [e + A' ,1 - e - A']. Note that for not close to the 

boundaries of this set, the partially- minimized coverage is achieved at a value of 
{Pi:Pi:P2:P2) close to the bouudarics of A. The large-sample analysis described 
in Appendix C includes the test statistic T which has an N{X, 1) distribution, where 
A = (^1 — (^2)/^/crf + (7|. In this partial minimization, A is a function of 5 e 
[—A, A], for each given In Appendix D, we show that the range of this 

function includes the interval [—2,2]. Therefore, this partial minimization includes 
the consideration of a wide interval of values of A, suggesting that the partially- 
minimized coverage will be quite small. Of course, whether or not this is, indeed. 




r = 
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the case needs to be assessed numerically. 

Consider the case-control study data described in Section 3, which consists of 
two strata with sample sizes rii — 1092, n[ — 467, n2 — 449 and n'a = 488. In 
this case, A = 0.056062 and r = 1.333316, so that A' = 0.074748. Figure 1 is 
a contour plot of the partially minimized coverage probability, as a function of 
{pi,p[) in [0.02 + A, 0.98 - A] x [0.02 + A', 0.98 - A'], for 1 - a = 0.95. Figure 
1 demonstrates that the large-sample approximation to the simultaneous coverage 
probability is much less than 0.95 for (pi,p'i,P2,P2) from the boundaries of the 
parameter space A — [0.02, 0.98]^. 

It is straightforward to show that the harmful effect of the prehminary test of 
homogeneity of the stratum-specific odds ratios does not disappear as the sample 
sizes increase. Consider m = 1092A^, n[ = 467A^, n2 = 449A^ and n'2 = 488A^, 
where N is a positive integer. It may be shown that, as we increase A^, the partially- 
minimized coverage probability converges to a limiting value for each {pi,p'i). The 
contour plot shown in Figure 1 does not differ greatly from the contour plot of 
this limiting value. In other words, the harmful effect of the preliminary test of 
homogeneity of the stratum-specific odds ratios does not disappear as the sample 
sizes increase. 
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0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 

Figure 1: Plot of the partially-minimized coverage probability of the confidence 
interval resulting from the two-stage analysis, with nominal coverage probability 
0.95 and nominal level of significance 0.05 of the preliminary test of homogeneity. 
This plot shows the partially-minimized coverage probability as a function of 

8. Discussion 

The literature on the effect of preliminary statistical model selection (using, for 
example, hypothesis tests or minimizing a criterion such as AlC) on confidence 
intervals begins with the work of Freeman (1989) who analyzed the effect of a pre- 
liminary test of the null hypothesis of zero differential carryover in a two-treatment 
two-period crossover trial on the confidence interval for the difference of treatment 
effects. This literature has grown steadily since this work of Freeman and is re- 
viewed by Kabaila (2009). It is commonly the case that prehminary model selection 
has a highly detrimental effect on the coverage probability of these confidence in- 
tervals. However, each case (specified by a model, a model selection procedure and 
parameters of interest) needs to be considered individually on its merits. 
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Our results show that the prehminary test of homogeneity of the stratum-specific 
odds ratios should not be used. The harmful effect of this preliminary test is very 
substantial and exists far from the boundaries of the parameter space. Furthermore, 
this harmful effect does not disappear with increasing sample sizes. 
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Appendix A: The search used to find an approximation to 
the minimum simultaneous coverage probability 

In this appendix we describe the search through the parameter space that was 
used to find an approximation to the minimum simultaneous coverage probability 
of specified confidence intervals for the log-odds ratios 9i and 02- As shown in 
Appendix B for the particular case P{9i e /i,^2 £ -^2), this simultaneous coverage 
probability is a discontinuous function of (pi,p'i,P2,P2)- This makes it difficult to get 
a very accurate estimate of the simultaneous coverage probability minimized over 
(Pi)P'i)P2,P2) the parameter space A. Nonetheless, the following search method 
provides a rough estimate of the minimum simultaneous coverage probability. 

For a given value of (pi,p'i,P2,P2)> "^^ estimate the simultaneous coverage proba- 
bihty of the confidence intervals by Monte Carlo simulation. We use a search method 
of the type described by Kabaila and Giri (2008) (cf. Section 3.1 of Kabaila and 
Leeb, 2006). Define the step length h — 0.096. The simultaneous coverage probabil- 
ity of these confidence intervals is estimated using M — 10000 simulations for each 
(Pi,Pi,P2,P2) belonging to the set (0.02, 0.02 + h, 0.02 + 2/i, . . . , 0.98)1 The 10 values 
of (pi,p'i,P2,P2) with the lowest estimates of this coverage probability are then se- 
lected for further consideration. For each of these 10 values, the coverage probabihty 
is then re-estimated using M = 200000 simulations. The value of {pi,p[,P2,P2) with 
the lowest estimate of this coverage probability is then selected for further consider- 
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ation. For this value, the coverage probabihty is then re-estimated using M — 10^ 
simulations. 

Appendix B: Discontinuity of P{Oi G A, ^2 G ^2) as a 
function of (^1,^1,^25^2) 

Consider the simultaneous coverage probability P(6'i e A, ^2 G -^2)- Let p = 
(Pi)P'i)P2,P2) y — (yi) yi) y2, Z/si)- ^^^o ^ denote the set of possible val- 
ues of y, so that y — {0, . . . , ni} x {0, . . . , n'l} x {0, . . . , 712} x {0, . . . , 712}. Now let 
ii{y) denote the interval li evaluated at observed value y {i = 1,2). Define B{p) to 
be the set of y belonging to 3^ such that 9i e h{y) and 62 G hiv), for given p. Note 
that 

p{e,eh,e2ei2)= ny = v)- 

yeB(p) 

Whilst P{Y = y) is a. smooth function of p for each y & y, the set B{p) changes as 
we change p. This leads to P{6i e /i, ^2 G h) being a discontinuous function of p. 

Appendix C: Details of the analysis using the large-sample 
approximation 

Note that ©1 and ©2 are independent random variables. The large-sample approx- 
imation described in Section 4 is as follows. The estimators ©i and ©2 have the 
following distributions: ©i ~ N{9i,af) and ©2 ~ A^(6'2,o"|), where af and o"! are 
known. We test the null hypothesis of homogeneity Hq : 9i — 62 against the alter- 
native hypothesis Hi : 9i ^ 62, using the test statistic 

Let /3 denote the level of significance of this test. Note that T ~ A'"(A, 1), where 
A = (^^1 — 62) /v^o"! + o"!- Define the quantile Cq by the requirement that P{—Ca < 
Z < Co) — 1 — a foT Z N{0, 1). We accept Hq if \T\ < c^; otherwise we reject Hq. 
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If Hq is rejected then the confidence intervals for 9i and 62, with nominal simul- 
taneous coverage 1 — a, are Ii and I2 respectively, where 

with Ca defined hy P{- 0^ < Z < Ca) ^ v^T^ for Z ~ A^(0, 1). Define 

which is the estimator of 9, assuming that 9 = 9i — 62. li 9 — 9i = 92 then the 
following confidence interval for 9 has coverage 1 — a: 



J 



1 1 

2 ~^ 2 



-1/2 



, + Ca 



1 



1 



2 ~^ 2 



-1/2- 



Let Ji — J and J2 = J. If i^o is accepted then the confidence intervals for 9i and 
92, with nominal simultaneous coverage 1 — a, are Ji and J2, respectively. 

Our aim is to evaluate the coverage probability of the simultaneous confidence 
intervals for 9i and ^2 resulting from the above procedure for given rii, pi and p[ 
{i — 1,2). By the law of total probability, this coverage probability is equal to 

P{9^ eJ,92e J, \T\ < c^) + P{9, e /i, ^2 e h, \T\ > c^). 

To evaluate this coverage probability, we will make use of the following readily- 
established results. The first result is that © and ©1 — ©2 are independent random 
variables. Since ©i and ©2 are independent normally-distributed random variables, 
(0, ©1 — ©2) has a bivariate normal distribution. We therefore prove this result by 
showing that Cov(0, ©i — ©2) = 0. It is a corollary of this result that and T are 
independent random variables. Now {9i G J} = {© G Ki}, where 



9i - Cn 



^2 ^2 



-1/2 



, 9i -\- Ca 



,-2 ^2 



-1/2' 



Thus 



P{9i G J, ^2 e J, \T\ < Cp) = P(0 G i^i, G K2, \T\ < c^) 

= P(0GXi,©GX2)P(|r| <c^) 
^P{QeK,nK2)P{\T\<cp). 
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Let a denote the maximum of the lower endpoints of the intervals Ki and K2. Also 
let b denote the minimum of the upper endpoints of Ki and K2. Observe that 

PieeK,nK2) = i\ . , 

I < © < otherwise. 
We compute P{a < Q < b) using the fact that 6 ~ N{9aviw)^ where 

1 



^"'^ " (IK) + ^""^ " 

By the law of total probability, 



P(^i e /i, ^2 e /2) = P{h e /i, ^2 e \T\ > c^) + P{e, e /i, ^2 e h, \T\ < c^). 

Since P(^i e /i, 62 e h) = I - a, 

P(^i e /i, 02 e /2, |T| > c^) = 1 - a - P(^i e /i, ^2 e /2, |T| < c^). 

We compute ^(6*1 e /i,6'2 € /2, \T\ < cp) using the following method. Straightfor- 
ward manipulations show that this probability is equal to 

P\ -Co, <Zi < Ca, -Ca < Z2 < Ca, 



^1 - ^2 / ^ / (72 ^ , /, , ai e, - 02 



<Zi < _Z2 + C^W1 + ^ 



where Zi = (0i — 6i)/ai and Z2 = (02 — 02)/<^2- Since Zi and Z2 are independent 
and identically A'"(0, 1) distributed, this probability is equal to 



(l){zi) dzi 0(^2) dz2 
where (j) denotes the A^(0, 1) probability density function and 



(CI) 



B{Z2) = LinL2{z2) 
-^2(^2) = 



— Z2 - C/3l/ 1 + 



61 — 62 (J2 



,—z2 + cp\n- + ^ 



h-62 

Ox 
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Let a{z2) denote the maximum of the lower endpoints of the intervals Li and L2{z2). 
Also let b{z2) denote the minimum of the upper endpoints of the intervals Li and 
L2(z2). Let 



g{z2) = / (f){zi) dzi. 

'B(Z2) 



if d{z2) > h{z2) 

$ (6(2:2)) - (5(^2)) otherwise 



Observe that 

g{z2) 

where $ denotes the A^(0, 1) distribution function. Thus fICip is equal to 

giz2)(j){z2)dz2 (C2) 



Note that g{z2) is a very smooth function of Z2 G [— c^, Ca] , except at a finite number 
(up to 4) values of Z2, where this function is continuous but does not possess a first 
derivative. Therefore, f lC2p is computed by adding the numerical integrals over the 
obvious sub intervals that have at least one of these values of Z2 as an endpoint. 

Appendix D: A property of the partial minimization of 
the approximate coverage considered in Section 6 

The large-sample analysis described in Appendix C includes the test statistic T 



which has an A^(A, 1) distribution, where A = (^^i — 62)/ a/cti + cri- the partial 
minimization described in the second paragraph of Section 6, A is a function of 
S G [—A, A], for each given {pi,p'i). In this appendix, we show that the range of 
this function includes the interval [—2,2]. 

Suppose that p2 = pi + 6 and P2 = p\ + 5', where \b\ and are small. By Taylor 
expansion, 

1 1 - 



y^^ Pi(l-P'l) Pl(l-Pl) 



1 1 \ 1 /I 



ni ^ n2y pi(l -pi) ^ ^ n'^J pi(l -pi 
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Now suppose that 5' — —rS, where r is defined in Section 6. Thus 



A 




ni 712/ -Pi) 




K pi{l-pi) 



5. 



Since pi{l - pi) < 1/4 and p[{l - p[) < 1/4, |A| > 2\5\/A, where A is defined in 
Section 6. 
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